CODEWITHSUNDEEP
jmeterperformance-testingresponse-time
Souvik
Souvik

Reputation: 51

Why is Average response time is reducing when we are increasing the number of users?

I am using J-Meter to run a performance test with different number of users. With 1 user, the avg response time is 1.4 seconds, but with more number of users, it's logical that the avg response time will go up, but instead it is reducing. Can anyone explain why? The test scenario is that I am interacting a few times (2-3 interactions) with a chat bot.

Please help me understand this confusing results below:

1 user   - 30 seconds  - 1.3 seconds  (average response time)
5 users  - 60 seconds  - 0.92 seconds (average response time) 
10 users - 60 seconds  - 0.93 seconds (average response time) 
20 users - 120 seconds - 0.92 seconds (average response time)

Upvotes: 0

Views: 875

Answers (1)

timbre timbre
timbre timbre

Reputation: 13980

First iteration of first user often involves some overhead on client side (most commonly DNS resolution), and can have some overhead on server side (server "warmup"). That overhead is not required in the following iterations or users.

Thus what you see as reduction in average time is actually reduction of the impact of the slower "first user first iteration" execution time on overall outcome. This is why it's important to provide a sufficient sample, so that such local spike does not matter that much anymore. My rule of thumb is at least 10000 iterations before looking at any averages, although level of comfort is up to every tester to set.

Also when increasing number of users, you should not expect average to be worse, unless you reached a saturation point: it should be stable rather. So if you expect your app to be able to support not more than 20 users, than your result is surprising, but if you expect application to support 20000 users, you should not have any average degradation at 20 users.

To test if this is what happens, try to run 1 user, but for much longer, so that total number of iterations is similar to running 20 users for example. Roughly you need to increase duration of test with 1 user to 20 min to get to similar number of iterations (i.e. same length of test would be 120 sec, but also x20 iterations with 20 users, giving you rough number of 20 min total for 1 user)

Upvotes: 0

Related Questions