Is there a technique to predict performance impact of application

Question

A customer is running a clustered web application server under considerable load. He wants to know if the upcoming application, which is not implemented yet, will still be manageable by his current setup. Is there a established method to predict the performance impact of application in concept state, based on an existing requirement specification (or maybe a functional design specification). First priority would be to predict the impact on CPU resource.

Is it possible to get fairly exact results at all?

Answer 1

I'd say the canonical answer is no. You always have to benchmark the actual application being deployed on its target architecture.

Why? Software and software development are not predictable. And systems are even more unpredictable.

Even if you know the requirements now and have done deep analysis what happens if:

• The program has a performance bug (or two...) - which might even be a bug in a third-party library
• New requirements are added or requirements change
• The analysis and design don't spot all the hidden inter-relationships between components
• There are non-linear effects of adding load and the new load might take the hardware over a critical threshold (a threshold that is not obvious now).

These concerns are not theoretical. If they were, SW development would be trivial and projects would always be delivered on time and to budget.

However there are some heuristics I personally used that you can apply. First you need a really good understanding of the current system:

• Break the existing system's functions down into small, medium and large and benchmark those on your hardware
• Perform a load test of these individual functions and capture thoughput in transactions/sec, CPU cost, network traffic and disk I/O figures for as many of these transactions as possible, making sure you have representation of small, medium and large. This load test should take the system up to the point where additional load will decrease transactions/sec
• Get the figures for the max transactions/sec of the current system
• Understand the rate of growth of this application and plan accordingly

Perform the analysis to get an 'average' small, medium and large 'cost' in terms of CPU, RAM, disk and network. This would be of the form:

• Small transaction
  • CPU utilization: 10ms
  • RAM overhead 5MB (cache)
  • RAM working: 100kb (eg 10 concurrent threads = 1MB, 100 threads = 10MB)
  • Disk I/O: 5kb (database)
  • Network app<->DB: 10kb
  • Network app<->browser: 40kb

From this analysis you should understand how much headroom you have - CPU certainly, but check that there is sufficient RAM, network and disk capacity. Eg, the CPU required for small transactions is number of small transactions per second multiplied by the CPU cost of a small transaction. Add in the CPU cost of medium transactions and large ones, and you have your CPU budget.

Make sure the DBAs are involved. They need to do the same on the DB.

Now you need to analyse your upcoming application:

• Assign each features into the same small, medium and large buckets, ensuring a like-for-like matching as far as possible
• Ask deep, probing questions about how many transactions/sec each feature will experience at peak
• Talk about the expected rate of growth of the application
• Don't forget that the system may slow as the size of the database increases

On a personal note, you are being asked to predict the unpredictable - putting your name and reputation on the line. If you say it can fit, you are owning the risk for a large software development project. If you are being pressured to say yes, you need to ensure that there are many other people's names involved along with yours - and those names should all be visible on the go/no-go decision. Not only is this more likely to ensure that all factors are considered, and that the analysis is sound, but it will also ensure that the project has many involved individuals personally aligned to its success.