CODEWITHSUNDEEP
performancejmeterperformance-testingloadrunnerneoload
ram kumar
ram kumar

Reputation: 35

Overloading Load Generators will impact response times of transactions?

I know overloading of load generators will impact execution times and volumes(No.of transactions per hour) but I am not sure whether it will impact response times as well, my hypothesis is it won't impact the response times, if i am wrong please let me know how it will impact response times?

Upvotes: 1

Views: 1775

Answers (4)

James Pulley
James Pulley

Reputation: 5682

Your hypothesis is incorrect. An overloaded load generator will impact response times because the speed of execution of everything on the host slows, including the virtual users. Even turning up log levels for your virtual users will slow your response times as this introduces a disk constraint and arbitration of the disk write heads amongst dozens/hundreds of competing processes writing logs to the disk - which is why it is recommended to only use "write on error" as your log level under test. Immature test tool users tend to write poor test code, which hogs CPU and Memory resources, which in turn leads to fewer users you can run on a host without impacting the performance of the users.

There are a number of rules of thumb which can help you to identify an issue related to load generator influence. The first of these is the use of a control factor in your testing. If you recall, a control factor is an element in your test design which allows you to measure the integrity of the test itself, independent of what is being tested. In the case of a performance test you can introduce either a control application, which will run side by side with virtual users for the duration of your test or a control load generator.

With a control application you only need to include a handful of users on each load generator for the control application. These users run for the duration of the test. Their response should be as a constant, not degrading due to the low level of load on the control application. If you do observe a degradation, a raising of response time, in your control group then you you have a generator induced influence on response time. You have an overloaded load generator.

In the second case, a control generator, you have all hardware matched load generators and you are running a single virtual user of each type on a control load generator, a very lightly loaded control generator. During the test observe the response times for the control load generator versus the remainder of the population of a similar type. If both the control load generator group and the global group ( other load generators) degrade at the same rate then you have an application induced performance issue with high confidence in your results. If you observe that the global times are degrading but the control group is not then you have an issue with overloaded load generators causing a degradation of local virtual user performance.

In either case it is recommended that you use at least three load generators in addition to your controller node. With the control application you would be looking at a balanced load across the minimum of three. With the control load generator you would be looking at two for primary load and one for the control load. Keep in mind that three is a minimum number of load generators. Depending upon your load and your technology stack you could be looking at 100's of load generators that you need control elements to be a part of.

Other rules of thumb: Never exceed 75% of the available resource pool for any given finite resource on your load generator (CPU, DISK, MEMORY, NETWORK) or your ring 0 resource hits will drive your virtual user performance lower as the core resource has to be serviced by the operating system. This leads to an assumption that you monitor your load generators just as you are monitoring your application under test.

Minimize logging. Watch for swapping on your virtual user processes, as all tool manufacturers have marked their test tool virtual users to be eligible for swapping. You take a disk hit and a memory hit when this happens. Bad magic for slowing your users down independent of your application under test.

Upvotes: 1

Joel Deutscher
Joel Deutscher

Reputation: 170

Response times will be impacted by an overloaded load generator. The load generator is in charge of sending the request, receiving the response and processing the response (validation, etc). It’s the sending, receiving and processing that can all be impacted by an over saturated load generator, resulting in lower throughput, though its primarily the processing that is the issue.

If the tool is delayed in registering that the response has been received this can impact the reported response times and report slower response times than the actual. This is made worse when the request is trying to download additional resources (CSS, JS, Images) as this can multiply the issue.

Upvotes: 1

Vadim Kleyzit
Vadim Kleyzit

Reputation: 812

When comparing overloaded and not overloaded load generator, the first one will register reduced response time as it will generate reduced hit rate because of injecting processing delays when issuing requests. In other words, overloaded load generator will incorrectly report higher performance. This will be true only if in both tests your application does not incur performance errors leading to failing transactions which complete faster.

Upvotes: 2

UBIK LOAD PACK
UBIK LOAD PACK

Reputation: 34526

It will impact your response times as well whatever the tool you use, some examples:

  • cpu overloading will make the tool process responses slowlier impacting at least full loading time
  • network overloading will impact response times as a contention will occur
  • garbage collection will also impact it ...

So rule of thumb, never ever have an overloaded injector.

Upvotes: 2

Related Questions