Performance testing on VSTS

Question

I am trying to figure out the mechanics of the performance test suite we have in VSTS 2010.Currently I have a couple of unit tests that I need to simualte under a 60-70 req/sec load,which I am unsure how to.I need to be able to calculate the roundtriptime it took for each request.I am currently unsure how to do this I have tried creating a load test and running the test but things like the:

1. Transactions/Sec
2. Avg. Transaction Time (sec)
3. Pages/Sec
4. Avg. Page Time (sec)
5. Requests/Sec
6. Requests Failed
7. Requests Cached Percentage
8. Avg. Response Time (sec)

for the overall results are not being populated when I am through with the result.The System I am testing is a WCF rest service,and I have a few tc's that genereate a request and call the resource.Any clues and directions are appreciated thanks!

Answer 1

I am sure you might have got the answer to this till now. But I am posting this answer for larger community. Since I also faced similar issues.

Question: How to get the transactions wise response time details. Since Visual Studio does not give transaction details for load tests with unit tests? Referring to your question 1-2.
--> There is a TestContext.BeginTimer Method in C#. If you use that around your actual WCF call,transaction level details will be seen on results. This way you can create round trip time.

Question: You want to generate load at about 60-70 req/sec. How to do that?
--> As you might know, there are three load patterns in visual studio.
1. constant load pattern
2. step load pattern
3. Goal-based load pattern

you might want to go with Goal-based load pattern, where you have to apply transactionName and goal in-terms of Transactions/Sec. In this case you can apply 60 and 70. Visual Studio will increase or decrease the userload based on the transaction per second and try to remain at defined load level. I hope this helps.

Answer 2

In case of VSTS performance testing when we are creating a Unit test and executing it with LoadTest using controllers and agents, it will not give information like Transactions/Sec Avg. Transaction Time (sec) Pages/Sec Avg. Page Time (sec) Requests/Sec Requests Failed Requests Cached Percentage Avg. Response Time (sec)

as this is not a webpage for which we are doing performance testing, here we had created a unit test which is consuming the exposed methods of webservices. it will show you only stats like Avg. Test Time, Threshold voilations, total tests in Trend-report. let me know if more information needed.

Answer 3

I think that many of the items in the list are only provided by Visual Studio's Web Performance Tests, not by unit tests. Load Tests are intended to use Web Performance Tests but they can also call unit tests and Coded UI tests.

To get the performance measurements you want you could create a Web Performance Test. Recording such a test within Visual Studio may be possible if you have a web page that provides access to the WCF service. I have not tried this route.

You should be able to create tests of WCF services by creating their requests. First create an empty Web Performance Test then use the "Insert Web Service Request" from the context menu or icons. See How to: Create a Web Service Test for more details of this. The program Fiddler2 can also be used to record a call of the WCF service and that can be exported to create a Web Performance Test.

Answer 4

It sounds like you're on the right track, but you could have missed any number of things. Here's a brief checklist:

1. Make sure your tests are running correctly and successfully.
   • Make sure your WCF configuration is correct.
   • Make sure any connection strings are correct.
2. Ensure that you have added the unit tests to the load test. The load test wizard is fairly simple & intuitive for setting up a basic load test.
3. Ensure that you have some users running in the load test (i.e. you aren't running 0 users). There are a number of different 'test types' you can run, & depending on the type you choose the configuration will vary.
4. Check that the load test is running. By this I mean check to see that the service is being hit & work is being done. (e.g. check the database being updated, or log something to a file etc)

This link might help (http://visualstudiomagazine.com/articles/2010/07/08/load-testing-with-visual-studio-2010.aspx) although I expect you've already tried google.

One thing I've noted is that it is often helpful to have the tests sleep for a short period. e.g. I've had difficultly with 2 users each running 50 tests per second, but more success with 100 users which each execute once and then sleep for a second. The same throughput but for some reason the test rig held up better.