Intermittent '401: Unauthorized' exceptions from server

Question

I have a web service that is hosted in a clustered environment. The web application that is calling this service is also hosted in a clustered environment, but on a different set of IIS6 servers. Hence, the application servers are appserv1 and appserv2 and the service servers are svcserv1 and svcserv2. We don't control which servers are accessed since we typically just refer to them as either appserv or svcserv, respectively.
The service is a WCF Service but has been created to be compatible with the .Net 2.0 Web Service framework. The application runs fine, when it runs, but probably > 35% of the time the service responds with an Exception: The request failed with HTTP status 401: Unauthorized. error.

I've seen others that recommend setting the credentials directly and my application is doing this as follows.

Dim cc As New CredentialCache()
Dim service As WCFServiceRef.Reports

service = New WCFServiceRef.Reports

service.Url = serviceURL
cc.Add(New Uri(service.Url), "Negotiate", New NetworkCredential("username", "password"))
service.Credentials = cc

reportData = service.GenerateReport(reportid, True, parameters, "PDF", Environment)

I have also tried to directly access the individual servers by changing the reference URLs to bypass the load manager and go directly to the domain name for the server but this hasn't made any difference.

I have also seen this MSDN KB article but since I don't directly have access to the server configuration (and it's difficult to get anything changed) I wanted to be sure there isn't something I can do from the application side. Note that the server has been configured for Windows Authentication and does not allow anonymous access.

Thanks!

Answer 1

Another thing to consider:

Do you need sticky sessions? Are there servers that are recycling too often?

There's a whole bunch of reasons why you may be getting 401s, so you'll just have to dig around and find out what's going on.

Answer 2

A lot of this depends on whether your authentication attempts are using Kerberos or falling back to NTLM. I would suggest using a tool like Fiddler to capture packets being sent from your app server to validate what authentication protocol is being used.

If you find you're using Kerberos, here's a few things to try:

1. Get your admins to enable Kerberos error logging on both servers, then reattempt the authentication and check for any Kerberos errors in the event logs. See http://support.microsoft.com/kb/262177.
2. Check the SPNs registered for your service - assuming the FQDN of your service is myservice.prd.ad you would want two SPNs - they should be registered to the servers hosting your service if your IIS6 app pool is not running as a service account, and should be registered to the service account otherwise:
   • HTTP/MYSERVICE.PRD.AD
   • HTTP/MYSERVICE
3. Also ensure that any DNS entries registered for your service are A (hostname) records rather than CNAME (alias) records. I've encountered some issues in environments I've worked in where XP/2003 machines attempted to get a Kerberos ticket for the wrong SPN when CNAME records were used for DNS (whereas Windows 7/2008 machines did not).
4. You could also try installing DelegConfig on your service servers, it's very good at sorting out the most common issues people have with Kerberos authentication. See http://www.iis.net/community/default.aspx?tabid=34&g=6&i=1887

Hopefully that helps a bit. If you're using NTLM then I unfortunately don't have many ideas as I'm used to working in Kerberos-only environments.

Answer 3

First thing is to check the IIS logs on each machine and see if the 401 errors are coming from a single machine.

The next thing to check is if the 401's are related to specific urls.