CODEWITHSUNDEEP
c#windowskerberosclouderawebhdfs
xadzax
xadzax

Reputation: 21

Windows WebHDFS Client to Cloudera Hadoop

We have a Windows application which is communicating fine via the WebHDFS Client (In the Incubator phase) http:/ /hadoopsdk.codeplex.com/wikipage?title=WebHDFS%20Client&referringTitle=Home to a Cloudera Hadoop installation. The next phase is to establish Kerberos authentication via http. I am having difficulty finding much on this topic between a Windows client and a Linux/Apache server.

Most of the examples I've seen are using cURL --negotiate as the mechanism : http://hadoop.apache.org/docs/r1.0.4/webhdfs.html#Delegation+Token+Operations

Everything else I've found for .Net has been really low level http://msdn.microsoft.com/en-us/library/ms995331.aspx

Is there anything out there that I can use or am I going to have to write custom code?

Upvotes: 0

Views: 1462

Answers (1)

xadzax
xadzax

Reputation: 21

I found the solution to my problem being that I misunderstood how Kerebros SNEGO was implemented.

For those of you that are in the same predicament I hope this helps..The authentication is done between the client (Windows machine) and the kdc (Linux) at the time of the users logon of the client (for one configuration). After the ticket has been issued WebHDFS communication can be established in a more secure manner. This quote is from the Cloudera team (which were extremely helpful BTW).

"If the Cloudera cluster is kerberized then WebHDFS on the server side will accept SPNEGO Kerberos authentication. If the cluster is kerberized directly in AD realm and if the end user has logged into the same AD domain, that should be enough if the Microsoft clients supports it. Otherwise they either have to authenticate to the same kerberos realm with a MIT kerberos client and kinit or one way trust should be setup between the cluster kerberos realm and the AD realm where the endusers are authenticating."

Upvotes: 2

Related Questions