Test Duration when we execute JMeter Test with 1 user and more

Question

I have concerns about the duration of executing JMeter Test Scenario.

In fact, if we execute a manual test with 1 user in 1 hour, we will find the same duration or little more with 3 user. But, with JMeter test the duration with 1 user will be multiplied by the number of users.

During my scripting with this tool, I've noticed that JMeter always wait for the response of the request to pass to another request. It's like we have 1 user doing the work of multiple users.

Does any body have explanations about this issue?
Can we configure JMeter to perform like we have x users working in // ?

Answer 1

Depends how you are simulating users..

Each thread will wait for a response, but users are represented by different threads, and threads do not wait for each other to complete samples, they are independent.

If you are using threads in thread group to represent users (which is the intended usage), the threads will start concurrently and run independently, so you should see the 3 users complete in an hour, as with manual tests. If this is not happening, then you should check the resources used on the client running jmeter during single thread and multi thread runs. Or perhaps the bottleneck may be in the transport (ie bandwidth). All you have determined so far is that the bottleneck is prior to the server, you need to determine where it is.

It is also possible you have created a thread group for each simulated user. If this is the case you can set the groups to run consecutive or concurrent. The setting for this is on the Test Plan element at the very top of the tree. You want concurrent, so untick 'Run Thread Groups Consecutively'.

So it comes down to how you are simulating users.

Answer 2

In ideal world if you test your site with 1 user all set of actions completes in 1 hour. If you add another 100 users - still one hour, 1000 users - 1 hour.

In real world server response time increases with the number of concurrent threads. But it isn't something like

• 1 user - average response time 10ms
• 2 users - 20 ms
• 3 users - 30 ms

But

• 1 user - 10 ms
• 2 users - 10 ms
• 3 users - 10 ms
• 10 users - 11 ms

The whole idea of the performance testing is to determine the upper limit of server load and identify bottlenecks and issues which happen under the load so you could state something like: using this hardware and this configuration the application is capable of serving 500 concurrent users without serious delays. In case of 1000 users response time will be more than 15 seconds.

In regards to threads, JMeter thread starts, executes samples one by one for defined amount of loops and exits. It is applicable for each thread which are independent. Basic throughput control can be done via ramp-up and loops count (see Thread Group documentation on details)

More advanced load scenario definition can be done using following Test Elements:

• Ultimate Thread Group - which provides easy visual way of defining ramp-up, ramp-down and time to hold the load
• Constant Throughput Timer - to set desired load rate in requests per minute
• Synchronizing Timer - to pause all the test threads and release them at the same moment to produce a "spike"