WCF Clients performance issues

Question

I have an application that should be able to send hundreds of concurrent request to my wcf service. The service is exposing http federation binding.

Currently I am using a single client and TPL to send requests on multiple threads. I have also changed the system.net max connections setting to 1000 so that windows is not restricting wcf to atmost 2 concurrent requests.

I can see that the initial request take more time because they are getting the authentication token etc, and then the request time generally begins to reduces significantly, but then I see some spikes intermittently, which do not correlate with the server logs i have.

I am wondering if this is the best way to approach a high throughup highly scalable client.

I have tried using sharing Client proxies among multiple threads and sharing ChannelFactory among multiple threads. All threads are done using TPL.

The binding is shown

<basicHttpBinding>
        <binding name="Binding1" openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:05:00" closeTimeout="00:01:00" allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard" maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647" messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered" useDefaultWebProxy="true">
          <readerQuotas maxDepth="32" maxStringContentLength="2147483647" maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647"/>
          <security mode="TransportWithMessageCredential">
            <transport clientCredentialType="None" proxyCredentialType="None" realm=""/>
            <message clientCredentialType="UserName" algorithmSuite="Default"/>
          </security>
        </binding>
      </basicHttpBinding>

Answer 1

You mention you are using a single client instance and then fanning out work using TPL. WCF client instances are technically not thread safe. The cost of creating a client is extremely cheap and you should not be afraid of creating new instances so long as you Close() them when you're done.

Next, you should not be using TPL alone to get you the concurrency. When you're making WCF calls, it's all about network I/O and waiting on the response from the server. What you need to make sure you're doing in your client is using async WCF contracts. You can then combine these with TPL's Task.Factory.FromAsync to chain together any other workflow you might need to perform after the call returns. For example:

[ServiceContract]
public interface IMyService
{
    [OperationContract(AsyncPattern=true)]
    IAsyncResult BeginDoSomething();
    SomeResultType EndDoSomething(IAsyncResult result);
}

// Somewhere in the client app you store your channel factory (these are "expensive", these you cache)
ChannelFactory<IMyService> channelFactory = new ChannelFactory<IMyService>();

public void MyClientMethod()
{
   // Create a client channel
   IMyService myServiceChannel = channelFactory.CreateChannel();

   // Use TPL's FromAsync to invoke the async WCF call and wrap that up with the familiar Task API
   Task<SomeResultType>.Factory.FromAsync(myServiceChannel.BeginDoSomething
                                          myServiceChannel.EndDoSomething,
                                          null)
                               .ContinueWith(antecdent =>
                               {
                                    try
                                    {
                                        // NOTE: exception will be thrown here if operation failed
                                        SomeResultType result = antecedent.Result;

                                        // ... continue processing the result ...
                                    }
                                    finally
                                    {
                                        // NOTE: depending on your configuration you might want to watch for errors and .Abort() here too
                                        ((IClientChannel)myServiceChannel).Close();
                                    }
                               });      
}

All of your scalability is going to come from using the async I/O on that client call.

Answer 2

Some production performance issues with both WCF services and using the WCF clients (ClientBase or ChannelFactory directly) are due to the .NET garbage collector being forced to do a lot of work because of sloppy object disposal. I'd use Performance Monitor to check the behavior of the .NET garbage collector at runtime to see if this is causing the spikes.

If it is a GC issue then you need to do a review of your service and client code. On the service, ensure object instance scope is as limited as possible. This helps ensure instance are mostly garbage collected in Gen0 which occurs most often. The client code should also be reviewed but it is particularly important in the service code.

Also, in both the service but particularly on the client, make sure object instances that implement IDisposable are properly wrapped in a using statement EXCEPT for the WCF client instance. To handle the proper disposal of the WCF client, look at this brief blog post for a good pattern. This can get tricky if you are using a dependency injection container. You can search for "WCF do not use using" for more detail information on why. The BasicHttpBinding and some configurations of the WsHttpBinding can handle sloppy disposal but any binding that uses sessions will be prone to the GC issues.