WCF async Calls Do Not Return Immediately

Question

I'm using a loop to call WCF async functions.

for (int i = 0; i < listData.Count; i++)
{
    // Some in-memory process here

    Globals.CXClient.ComputePerformanceDimensionAsync(i); 

    // Some GUI process here
}

For whatever reason, the call to the async function always gets long delays around 10th iteration (i = 9). The delays depend on the system. If my WCF service is located at the same machine, the delay can be up to 20 seconds. If it is across LAN, it can be up to 40 seconds. If it is across Internet, it can take minutes. During the delay, the GUI freezes and nothing (e.g., the progress bar) is moving.

During the delay, the async processes continue to run. When the main thread finally gets the control after the delay and the GUI resumes moving, the WCF processes already proceeded to certain points. This means that the WCF services are not affected. It seems to me that the communications between the client and the server are held up.

After the first delay, the system seems to be "clogged" in the sense that the main thread constantly gets stalled for a few seconds before gaining the control back from the async calls.

I'm using wsHttpBinding. This is how it looks in web.config at WCF side:

<bindings>
    <wsHttpBinding>
        <binding maxReceivedMessageSize="2147483647" openTimeout="00:30:00" closeTimeout="00:30:00" sendTimeout="00:30:00" receiveTimeout="00:30:00">
            <readerQuotas maxStringContentLength="2147483647" maxArrayLength="2147483647" />
            <security>
                <message establishSecurityContext="true" />
            </security>
        </binding>
    </wsHttpBinding>
</bindings>

Any helps/hints/suggestions are very much appreciated as I'm stuck right now not knowing what to do next.