CODEWITHSUNDEEP
multithreadingasynchronouskestrel
Adrian Alexandru
Adrian Alexandru

Reputation: 41

Kestrel limiting requests concurrency

Because i'm struggling with thread starvation symptoms like i've decided to rewrite the whole calls chain (I/O, db paths) in async/await mode. After huge effort to rewrite async/await my terrible surprise was that has no effect on medium/high load. So i decided to reduce the test to the following sequence:

WebStressTool -> IIS(reverse proxy) -> Kestrel(out of process) -> await MiddlewarePipeline.Next() -> async Controller.Action -> await db.StoredProcedure(sql: WAIT FOR 1000ms)

The processing time for every request should be near 1000ms in full async/await calls chain. I expect that as the requests/second increase the response time should remain the same (1000ms). Here is the result:

  1. 1-9 rqs/sec -> ResponseTime: 1010ms. (WebStresTool: 9 rqs/sec). Its prefect,as expected
  2. 15 rqs/sec -> ResponseTime: 2200ms. (WebStresTool: 9 rqs/sec). Bottleneck
  3. 25 rqs/sec - ResponseTime: 4500ms. (WebStresTool: 9 rqs/sec). Bottleneck ....and so on

as the requests /sec increase the response time increase exponentially. The normal behavior in a full async/await scenario should be : as the increase or rqs/sec the response time should remain the same : time = 1000ms (Scalability).

So the symptoms are not related to thread starvation but to some bottleneck issue somewhere... who knows.

After more tests i discovered that the issue appears only when run under IIS (reverse proxy). The application threads not exceed 40 and has huge response time with load. Repeating the test without IIS (just kestrel server) the result was a success. The application is scalable.

It seems the runnig of kestrel behind IIS is the issue

Any suggestions, ideas, hints etc would be highly appreciated.

Best regards

Upvotes: 1

Views: 448

Answers (1)

Adrian Alexandru
Adrian Alexandru

Reputation: 41

Well i have a terrible suspicion : running the stress scenario on my development machine is limited by the hidden IIS own concurrency limitation. Only on windows server has no limitation. This means that i spent hundreds of hours to improve the response time, rewrite hot paths async, caching etc and i was misguided by this hidden limitation. Now i'm moving the stress test on windows server 2012. I'll be back soon

Upvotes: 2

Related Questions